The present invention relates to image correlation and, in particular, it concerns a platform with pseudo-autonomous image registration.
It is known to perform image registration between images viewed at different viewing angles. One example of particular interest is registration of a perspective image with an orthogonal photographic representation which allows features in the perspective image to be associated with their geographic locations. The techniques for performing such registration per se are well known in the art and will not be dealt with here in detail.
The following patent documents, mentioned as illustrative of applications of image registration techniques, are hereby incorporated by reference for all purposes as if fully set forth herein. Israeli Patent No. 111069 to Sroka et al. describes a method and system for marking object images acquired at high zooming by an airborne imaging device on a reference image having a wide field of view of the scene. U.S. Patent Application Publication No. 2002/0196248 to Kraus et al. describes a method for sharing visual information between a number of command and control (“C&C”) base units. Co-pending, co-assigned Israeli Patent Application No. 158685, which is unpublished at the date of filing this application and does not constitute prior art, extends these concepts to bi-directional communication between platforms via geographic coordinates which are represented in the local perspective view.
Turning now to FIG. 1, this shows a typical example of a scheme for implementing image correlation for an airborne platform with an imaging system, for example an unmanned aerial vehicle (“UAV”) 16. An image of a region of interest is derived from a camera having an imaging sensor 10 operated by a camera controller unit 14 and is transmitted via wireless transceivers 20, 22 to a base station 18, where it is processed by a processing system 24 and may be presented on a display 12 located at the base station or any other desired location. Camera controller unit 14 typically operates an optical zoom of imaging sensor 10 for selectively generating views with relatively wide field of view (FOV) and relatively narrow FOV. A “frozen” wide angle view is optionally displayed on a second display 26, preferably with a frame indicating the region of the zoomed view currently being displayed on display 12 to facilitate user orientation. A similar system and various additional preferred features for operation of the system are described in the aforementioned Israeli Patent No. 111069.
Each base unit 18 is also provided with reference data 30 corresponding to an orthogonal photographic representation of a region overlapping at least part of the field of view of the perspective view of imaging sensor 10. The orthogonal photographic representation is preferably associated with a defined geographic coordinate system, typically corresponding to the standard longitude-latitude or “north-south” coordinate system defined globally across the surface of the Earth. Finally in structural terms, the base unit 18 is typically in communication with other base units or other entities via a wireless data communication system, such as a data-link system, represented by transceiver 32.
FIG. 2 illustrates the relationship between the images/views received by the base unit 18. Specifically, there are shown a real-time detailed narrow FOV perspective image 40, a frozen wide-angle perspective image 42 and an orthogonal photographic representation 44. The wide-angle image is typically necessary to provide sufficient data for reliable correlation with the orthogonal photographic representation. As a result, correlation between the perspective view and the orthogonal photographic representation is performed as a two-step correlation as illustrated in FIG. 3. Specifically, the wide FOV image 42 is correlated with the orthogonal photographic representation 44 to generate a first mapping transformation T1, and the narrow FOV image 40 is correlated with the wide FOV image 42 to generate a second mapping transformation T2. The combination of mapping transformations T1 and T2 or their, inverses together fully define the mapping between the perspective image 40 and the orthogonal photographic representation 44 in both directions, for example, allowing a target 46 identified in perspective view 40 to be associated with a geographic coordinate in orthogonal representation 44 or the reverse.
It should be noted that the entire image correlation processing is performed at base unit 18 remote from the data gathering platform 16. This is necessitated by the heavy data processing and huge quantities of data typically involved in image registration of this type, rendering it impractical to perform the processing locally for each platform. Specifically, the data storage and processing requirements of correlating images to a geographical database typically require a server system with multiple parallel processors and corresponding instant-access mass-data storage systems. These requirements render it impractical for all but the most massive mobile platforms to carry a self-contained geographic-database image registration system. Furthermore, the need for continual updating of information in the geographical database encourages a centralized processing approach.
As a result of this centralized processing approach, each mobile platform is required to continuously download sampled images to the remote base unit site. This occupies a disproportionate amount of communication bandwidth, and renders the mobile platform completely dependent upon the base unit. The continuous computational load on the base unit also limits the number of such platforms which can be served by a given base unit.
There is therefore a need for a method and system which would provide pseudo-autonomous image correlation for a platform-based system.